System and method for representing virtual object information with haptic stimulation

ABSTRACT

A system for haptic stimulation includes a haptic device and preferably interfaces with and/or includes a virtual platform. A method for haptic stimulation includes: receiving a set of inputs; determining one or more parameters associated with a virtual object based on the set of inputs; and prescribing a stimulation pattern to one or more haptic actuators of a haptic device. Additionally or alternatively, the method 200 can include any or all of: presenting information to a user; selecting one or more haptic actuators of a haptic device based on the parameter(s); and/or any other suitable processes performed in any order.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.17/076,631, filed 21 Oct. 2020 which claims the benefit of U.S.Provisional Application No. 62/924,090, filed 21 Oct. 2019, and U.S.Provisional Application No. 62/972,540, filed 10 Feb. 2020, each ofwhich is incorporated herein in its entirety by this reference.

TECHNICAL FIELD

This invention relates generally to the sensory output field, and morespecifically to a new and useful system and method for representingvirtual object information in the sensory output field.

BACKGROUND

Haptic stimulation (equivalently referred to herein as tactilestimulation) has been shown to have several advantages in varioussensory use cases, such as: supplementing other forms of sensory inputs(e.g., audio, visual, etc.) in enhancing a user experience; replacingsensory inputs which might be compromised and/or otherwise unable to beperceived (e.g., audio for hearing-impaired individuals, visual mediafor visually-impaired individuals, etc.); and/or otherwise enhancinguser perception and conveying information.

Virtual platforms, such as virtual reality and/or augmented realityplatforms (e.g., for gaming, training, remote task performance, etc.)have begun to integrate haptic technologies, such as through the use ofhaptic gloves which attempt to mimic how objects would directly feel onthe hands of users. These gloves and/or other current systems, however,are conventionally very bulky, wired, and/or otherwise cumbersome touse.

The inventors have discovered a new and useful system and method torepresent information associated with virtual objects through tactilestimulation.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a schematic representation of the system 100 for representingvirtual object information with haptic stimulation;

FIG. 2 is a schematic representation of the method 200 for representingvirtual object information with haptic stimulation;

FIGS. 3A-3B depict a schematic representation of a user's hands relativeto a virtual object (virtual balloon) and a variation of wristbandhaptic devices.

FIGS. 4A-4C depict a variation of a user's interaction with a virtualobject and example embodiments of the stimulation patterns applied basedon the interactions.

FIG. 5 depicts a variation of a user's hand making contact with avirtual object and the resulting haptic stimulation applied to awristband device.

FIGS. 6A-6B depict a display of a virtual platform.

FIG. 7 depicts a user interacting with a virtual object in a virtualplatform.

FIG. 8 depicts a schematic variation of a user interacting with avirtual object in a virtual platform.

FIGS. 9A-9B depict a schematic variation of a user interacting with avirtual object in a virtual platform.

FIG. 10 depicts a variation of a haptic device.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

The following description of the preferred embodiments of the inventionis not intended to limit the invention to these preferred embodiments,but rather to enable any person skilled in the art to make and use thisinvention.

1. Overview

As shown in FIG. 1 , a system 100 for haptic stimulation includes ahaptic device 110 and preferably interfaces with and/or includes avirtual platform 120. Additionally or alternatively, the system 100 caninclude, define, and/or interface with any or all of: a processingsubsystem, control module, communication subsystem, user device, and/orany other suitable component(s). Further additionally or alternatively,the system 100 can include any or all of the systems, component,embodiments, and/or examples described in any or all of: U.S.application Ser. No. 14/750,626, filed 25 Jun. 2015, U.S. applicationSer. No. 15/661,934, filed 27 Jul. 2017, U.S. application Ser. No.15/795,054, filed 26 Oct. 2017, and U.S. application Ser. No.17/033,433, filed 25 Sep. 2020, each of which is incorporated herein inits entirety by this reference.

As shown in FIG. 2 , a method 200 for haptic stimulation includes:receiving a set of inputs S210; determining one or more parametersassociated with a virtual object based on the set of inputs S220; andprescribing a stimulation pattern to one or more haptic actuators of ahaptic device S250. Additionally or alternatively, the method 200 caninclude any or all of: presenting information to a user S230; selectingone or more haptic actuators of a haptic device based on theparameter(s) S240; repeating any or all of S210-S2150; and/or any othersuitable processes performed in any suitable order. Further additionallyor alternatively, the method 200 can include any or all of the methods,processes, embodiments, and/or examples described in any or all of: U.S.application Ser. No. 14/750,626, filed 25 Jun. 2015, U.S. applicationSer. No. 15/661,934, filed 27 Jul. 2017, U.S. application Ser. No.15/795,054, filed 26 Oct. 2017, and U.S. application Ser. No.17/033,433, filed 25 Sep. 2020, each of which is incorporated herein inits entirety by this reference.

The method 200 is preferably performed with a system 100 as describedbelow, but can additionally or alternatively be performed with anysuitable system or systems.

2. Benefits

The system and method for haptic stimulation can confer several benefitsover current systems and methods.

First, in some variations, the system and/or method confers the benefitof conveying information for the control (e.g., fine-grained control)and manipulation of virtual objects in a virtual reality and/oraugmented reality platform. Additionally or alternatively, the systemand/or method can enable any or all of: conveying information to theuser associated with the location of a virtual object outside of thefield-of-view of the user (e.g., behind the user); conveying informationassociated with phenomena not perceivable by a user (e.g.,echolocation), herein equivalently referred to as sensory addition;supplementing conventional sensory experiences with haptic stimulation(e.g., to enhance a gaming experience, to enhance a virtualteleconference experience, to enhance a virtual concert experience,etc.); and/or convey any other suitable information to a user and/orotherwise enhance a experience.

Second, in some variations additional or alternative to those describedabove, the system and/or method confers the benefit of utilizing and/orcreating a sensory displacement experience to efficiently and uniquelyconvey haptic information to the user. Sensory displacement can beenabled in the method 200, for instance, by receiving the hapticstimulation at a different body part and/or body region than where itappears visually in a virtual platform. In one set of variations, forinstance, a haptic device placed on the wrist provides hapticstimulation at the user's wrist which represents the interaction of auser's fingers with a virtual object. This can in turn function toenable the haptic device to be less cumbersome (e.g., smaller, placed ona more desirable and/or sensitive body region, placed on a lessconspicuous body region, wireless, etc.), maximize mobility and/ordexterity of the user such as at the body region visually portrayed tobe interacting with the virtual object (e.g., the user's fingers),and/or can confer any other suitable functions. In specific examples, ahaptic device placed on the wrist is used to convey haptic stimulationto the user's wrist which represents the interaction of the user'sfingers with one or more virtual objects, wherein the stimulation at thewrist is learned by the user (e.g., naturally, through training, etc.)to represent interactions of the user's fingers with a virtual object asdetectable visually by the user. This can be used in gaming, surgicaltraining (e.g., by enabling the user to maintain full hand mobility anddexterity), remote task performance (e.g., remote surgeries, remoterobot control, etc.), and/or any other suitable tasks.

Third, in some variations, additional or alternative to those describedabove, the system and/or method confers the benefit of enabling hapticstimulation to be applied to represent various different types and/orpotential features of virtual objects, such as, but not limited to, anyor all of: a virtual object's location relative to the user, atemperature, a texture, a pressure, a force, a discrete state (e.g.,within or outside of a particular threshold, alive or dead, hot or cold,safe or in danger, close or far, etc.), and/or any other suitablefeatures. In specific examples, the stimulation pattern associated withdifferent features (e.g., proximity vs. temperature) can be altered(e.g., in frequency, in amplitude, in a stimulation pulse pattern, etc.)to help the user distinguish between different types of informationbeing conveyed through haptic stimulation.

Additionally or alternatively, the system and/or method can confer anyother suitable benefits.

3. System 100

As shown in FIG. 1 , a system 100 for haptic stimulation includes ahaptic device 110 and preferably interfaces with and/or includes avirtual platform 120. Additionally or alternatively, the system 100 caninclude, define, and/or interface with any or all of: a processingsubsystem, control module, communication subsystem, user device, and/orany other suitable component(s). Further additionally or alternatively,the system 100 can include any or all of the systems, component,embodiments, and/or examples described in any or all of: U.S.application Ser. No. 14/750,626, filed 25 Jun. 2015, U.S. applicationSer. No. 15/661,934, filed 27 Jul. 2017, U.S. application Ser. No.15/795,054, filed 26 Oct. 2017, and U.S. application Ser. No.17/033,433, filed 25 Sep. 2020, each of which is incorporated herein inits entirety by this reference.

The system functions to apply haptic stimulation to a user based onvirtual object information and/or any other suitable informationassociated with a virtual platform (e.g., virtual reality [VR] platform,augmented reality [AR] platform, etc.). The virtual object informationand/or other information preferably includes visual information (e.g.,as perceived by a user on a display, as perceived by a user with a VR/ARheadset, etc.), but can additionally or alternatively include audioinformation, haptic information, conventionally non-perceivableinformation (e.g., echolocation, electromagnetic waves, ultrasound,non-visible light, etc.), olfactory information, and/or any othersuitable information. The system further preferably functions to providehaptic stimulation through sensory displacement, wherein the hapticdevice is arranged at and provides stimulation to one or more first bodyregions (e.g., wrist(s), ankle(s), etc.), wherein the informationconveyed through the haptic stimulation (e.g., visual information of avirtual object) is associated with one or more second body region(s)(e.g., hand(s), finger(s), leg(s), arm(s), etc.) separate and distinctfrom the first body region(s). This can confer numerous benefits (e.g.,as described above), such as, but not limited to, any or all of:enabling the first body region(s) to maintain full and/or maximalmobility and/or dexterity; reduce a form factor/footprint of the hapticdevice; place the haptic device on a more convenient location and/orsimpler anatomy; enable the device to be wireless; reduce a requiredsurface area and/or complexity (e.g., number of haptic actuators,placement of haptic actuators, etc.) of the haptic device; enable hapticstimulation to be applied to a particularly, adequately, and/ormaximally sensitive body region of the user (e.g., able to interpretand/or distinguish haptic stimulation); and/or can confer any othersuitable benefits.

Additionally or alternatively, the system 100 can function to performany or all of the processing involved in transforming the informationinto one or more haptic stimulation patterns, and/or updating any or allof the information (e.g., visual information provided to user atdisplay) based on user inputs and/or motion.

Further additionally or alternatively, the system 100 can perform anyother suitable function(s).

The system 100 is preferably utilized in the performance of the method200, but can additionally or alternatively be used in association withany other suitable methods.

3.1 System: Haptic Device 110

The system includes a haptic device 110, which functions to producehaptic stimulation with which to stimulate and/or convey information toa user. Additionally or alternatively, the haptic device 110 canfunction to

The haptic device includes a set of haptic actuators 112, whichindividually and/or collectively function to apply haptic stimulation(equivalently referred to herein as a vibratory stimulation and/ortactile stimulation) to a user (e.g., at the first body region of theuser). The haptic stimulation preferably functions to convey informationto the user, but can additionally or alternatively function to enhance auser experience, provide entertainment to a user, and/or perform anyother suitable function(s).

The set of actuators 112 can include one or more of: an actuator (e.g.,linear resonant actuator [LRA], electroactive polymer [EAP] actuator,electromechanical polymer [EMP] actuator, etc.), a motor (e.g.,brushless motor, brushed motor, direct current (DC) motor, alternatingcurrent (AC) motor, eccentric rotating mass (ERM), etc.), apiezoelectric device, and/or any other form of vibratory element. Theset of actuators 112 preferably includes multiple actuators but canalternatively include a single actuator (e.g., a stationery actuator, atranslatable actuator, etc.). In variations including multipleactuators, the actuators can be of the same type (e.g., all LRAs, allERMs, etc.) or any combination of actuators (e.g., LRAs and ERMs). In afirst variation of the haptic device 110, the set of actuators includesa set of multiple (e.g., 2, 4, 6, 8, 10, between 2 and 10, between 2 and5, greater than 10, etc.) LRAs.

In haptic devices including multiple actuators, the actuators can bearranged in an array (e.g., 1-dimensional array, 2-dimensional array,3-dimensional array, etc.), arranged at least partiallycircumferentially around a body part (e.g., around a wrist, around halfof the circumference of the wrist, etc.), arranged along a body part(e.g., up and down an arm), arranged over a body region (e.g., over theuser's trunk, stomach, etc.), arranged among different body parts of auser (e.g., arranged around both wrists), and/or arranged in any othersuitable way. The vibratory elements can be directly coupled to the skinof a user (e.g., with an adhesive, with suction, etc.), separated from auser by a housing and/or fastener (e.g., a wristband), placed over auser's clothing, and/or coupled to the user in any other way. Invariations of the system configured to apply haptic stimulation to awrist of the user, the system preferably includes multiple actuators(e.g., 2, 3, 4, 5, 6, 7, 8, 9, 10, 15, 20, 25, 30, between 10 and 30,30, between 30 and 40, 40, between 40 and 50, 50, greater than 50, etc.)arranged around at least a portion of the circumference (e.g., half thecircumference) of the wrist. Additionally or alternatively, the systemcan include actuators circumscribing the entire wrist (e.g., 8 LRAactuators), and/or any other suitable number and arrangement ofactuators. In a set of specific variations, the haptic device includes 5actuators, wherein each of the actuators corresponds to a differentfinger/thumb of the user's hand. In specific examples, a haptic deviceis worn on each wrist of the user. Additionally or alternatively, theuser can wear a single haptic device, use/wear haptic devices on otherbody regions, and/or interface at any number of haptic devices at anysuitable locations/regions.

In a first set of variations, the body region(s) at which hapticstimulation is applied with haptic actuators of the haptic deviceincludes one or both wrists (e.g., partial circumference of wrists, fullcircumference of wrists, etc.) of the user. Additionally oralternatively, haptic stimulation can be applied at any or all of: oneor more hands of the user (e.g., in a glove form factor), one or morefingers of the user (e.g., in a glove form factor, in a ring formfactor, etc.), one or more arms of the user (e.g., in a sleeve formfactor, in an armband form factor, in a cast and/or sling form factor,etc.), one or more legs of the user (e.g., in a knee brace form factor,in a compression sleeve form factor, etc.), one or more feet of the user(e.g., in a shoe form factor, in a sock form factor, etc.), a torso ofthe user (e.g., in a vest form factor), a neck of the user (e.g., in ascarf form factor, in a neck wrap form factor, etc.), a head of the user(e.g., in a hat form factor, in a headband form factor, etc.), an ankleof the user (e.g., in an anklet form factor), and/or any other suitablebody part(s) and/or body region(s) of the user. Additionally oralternatively, any or all of these body regions can be interpreted ashaving received haptic stimulation (e.g., through sensory displacement,through haptic illusions, etc.), visually depicted as interacting withone or more virtual objects in a virtual platform (e.g., as describedbelow, as shown in FIG. 7 , etc.), and/or any other body regions caninterface with the haptic device 110 in any suitable way(s).

The haptic actuators are preferably individually and/or collectivelyconfigured to stimulate the user through one or more stimulationpatterns (e.g., series of stimulation patterns), wherein a stimulationpattern refers to the prescribed haptic stimulation applied at one ormore haptic actuators. Additionally or alternatively, any or all of theactuators can be configured to provide somatosensory stimulation via anyone of a number of methods, such as, but not limited to: vibration,pressure, squeeze, electrotactile stimulation, and/or any other suitablestimulation.

The stimulation patterns can prescribe any or all of the following tothe set of actuators (e.g., individually, collectively, etc.): amplitudeof vibration, timing of vibration (e.g., when to start, duration, whento end, etc.), sequence of vibration, identification of which of the setof actuators to vibrate, frequency of vibration, pulsing of vibration(e.g., timing of pulse, total duration of pulse, temporal spacingbetween pulse, duration of each pulse, frequency of pulsing, etc.)and/or any other parameter(s) of stimulation. The haptic actuators canbe configured to vibrate with any or all of these parameters fixed(e.g., fixed frequency, fixed amplitude, etc.), dynamic (e.g., dynamicfrequency, dynamic amplitude, dynamic duration, dynamic pulsing pattern,etc.), and/or any combination.

Any or all of the actuators can be configured to vibrate at any suitablefrequency or frequencies, such as any or all of: a fixed frequency(e.g., a single frequency); a range of frequencies (e.g., which can beadjusted and/or programmed according to the prescribed stimulationpattern, adjusted based on the vibration frequencies associated with theother actuators, etc.); and/or any other suitable frequency orcombination of frequencies. In some variations, the stimulation patternprescribes an amplitude of vibration and a duration of vibration to oneor more actuators of the set of actuators, wherein each of the set ofactuators is configured to vibrate at a fixed frequency. In specificexamples, each of a set and/or a subset of multiple actuators isassociated with a different fixed frequency, which can function tofacilitate interpretation of information conveyed through the hapticactuators. Additionally or alternatively, the stimulation pattern canprescribe a frequency of vibration, a dynamic pattern of vibration(e.g., alternating between actuators), and/or any other suitablecharacteristic or parameter(s) of vibration.

In variations having multiple actuators (e.g., placed in a linearfashion along a wristband, arranged in an array on a vest, etc.), thesystem can optionally be configured to reduce coupling between actuators(e.g., between adjacent actuators). This can be at least partiallyaccomplished by using actuators having different vibration frequencies(e.g., different single vibration frequencies, different programmedvibration frequencies, etc.) from each other, which can function toreduce mechanical coupling between actuators. In a first set of specificexamples, the actuation subsystem includes a set of multiple actuations,wherein each actuator (e.g., LRA) is associated with a differentprescribed frequency, such as a single prescribed frequency. This canfunction to increase the ease in which a user can interpret the location(e.g., at an actuator, in-between actuators for haptic illusions, etc.)and/or source of a haptic stimulus; help a user interpret informationand/or increase the amount of identifiable information being conveyedthrough haptic stimulation (e.g., associating particular phonemes withparticular frequencies, adding another variable with which todistinguish haptic stimulation, etc.); increase the degree to whichsignals dampen between adjacent actuators; and/or perform any othersuitable function(s). Additionally or alternatively, each actuator canbe configured to vibrate at multiple frequencies, but be programmed tovibrate at different frequencies (e.g., when multiple actuators arebeing actuated simultaneously, when applying a stimulation pattern, atall times, etc.). Further additionally or alternatively, the actuatorscan be configured to vibrate at any suitable frequency or frequencies.

Each of the actuators is preferably configured to be actuated with avariety of different amplitude values (e.g., according to a prescribedstimulation pattern), which can function to adjust the intensity withwhich information is provided to the user. This can in turn function todifferentiate between different information, such as, but not limited toany or all of the following ways: providing higher intensity stimulationfor greater values of a parameter (e.g., as described below); providingdifferent intensities of stimulation for adjacent actuators; increasinga range of different haptic stimuli able to be applied to the user(e.g., by adding in an adjustable amplitude parameter); enabling hapticstimuli to be distinguished from each other; enhancing the ability ofthe system to provide entertainment to the user (e.g., through controlof intensities to represent and/or enhance a musical performance, tomatch the beat of a song at a concert, etc.); enable importantinformation to be easily detected (e.g., high amplitudes for moreimportant and/or immediate information); to enable the provision oftactile illusion stimulation (e.g., by actuating multiple actuators inrapid succession); and/or can perform any other suitable function(s).

Additionally or alternatively, each of the actuators can be any or allof: actuated with a single and/or constant amplitude (e.g., eachactuator actuates with the same amplitude, the actuators vibrate withdifferent but fixed amplitudes, etc.); an amplitude of stimulation isdetermined in response to controlling a frequency of vibration (e.g.,according to the spec of the particular actuators); and/or the actuatorscan be otherwise stimulated in accordance with any other suitableparameters.

The haptic device 110 can optionally include any number of fastenersand/or housings which can function to couple the haptic device to theuser. These can include, for instance, but are not limited to, any orall of: one or more bands (e.g., wristband, arm band, leg band, ankleband, belt, etc.), garments (e.g., vests, clothing, hat, pants, socks,gloves, etc.), fabrics, attachment mechanisms (e.g., buckles, magnets,hook-and-loop closures, etc.), and/or any other suitable materials.

The haptic device 100 can additionally include and/or interface with anyor all of: a power source and/or power storage (e.g., a battery), ahaptic driver, processing systems (e.g., CPU, GPU, memory, etc.), useroutputs (e.g., display, speaker, vibration mechanism, etc.), user inputs(e.g., a keyboard, touchscreen, microphone, etc.), a location system(e.g., a GPS system), sensors (e.g., optical sensors, such as lightsensors and cameras, orientation sensors, such as accelerometers,gyroscopes, and altimeters, audio sensors, such as microphones, etc.),data communication system (e.g., a WiFi module, BLE, cellular module,etc.), and/or any other suitable components.

In preferred variations, the haptic device 110 is a wearable device,such as a wearable device coupled to the user with a fastener.Additionally or alternatively, the haptic device 110 can be otherwisecoupled to the user, such as: placed against and/or placed on, held by(e.g., as a joystick and/or controller), wrapped around (e.g., as ascarf), resting upon, touching, secured to, arranged next to, arrangedunderneath (e.g., in a seat cushion), and/or otherwise coupled to theuser.

In a first variation, the haptic device 110 includes a set of multipleactuators coupled to a wristband, the wristband configured to be wornaround a wrist of the user, wherein the actuators are configured toapply haptic stimulation to the user's wrist based on informationassociated with a virtual object relative to a user's hand. In specificexamples, the wristband includes 5 actuators (e.g., LRA actuators),wherein each actuator corresponds to a particular finger/thumb of theuser. Additionally or alternatively, the wristband can include anynumber of actuators. Additionally or alternatively, in some specificexamples, the user wears a wristband on each wrist.

In a second variation, the haptic device 110 includes an array ofactuators coupled to a vest garment, wherein the haptic actuators cancorrespond to information associated with a virtual object relative tothe user's torso. Additionally or alternatively, the information can beassociated with any other suitable body regions of the user (e.g.,through sensory displacement).

In a third variation, the haptic device includes an array of actuatorsintegrated into one or more handheld controllers of a virtual platform.

Additionally or alternatively, the haptic device can include any othersuitable components (e.g., processing subsystem, communicationsubsystem, etc.) and/or be otherwise configured.

3.2 System: Virtual Platform 120

The system preferably includes and/or interfaces with a virtual platform120, which functions to provide and control virtual content (e.g., inVR, in AR, in mixed reality, in extended reality, etc.) including a setof virtual objects with which the user interacts during the method 200(e.g., as shown in FIG. 8 ).

The virtual platform can be configured for any or all of: gaming, taskperformance (e.g., remote surgery, remote control of robotics, etc.),simulation (e.g., military simulators, flying simulators), training(e.g., surgical training), immersive teleconferencing, and/or any othersuitable applications.

Virtual objects produced at the virtual platform herein refer to visualcontent/visual objects produced at the virtual platform which the usercan perceive at a display of the virtual platform, such as a display ina VR headset, and/or any other suitable display. The virtual objects canrepresent any or all of: inanimate objects (e.g., ball, balloon, weapon,house, building, chair, etc.), animals, people (e.g., virtualrepresentation of user, virtual representation of other users in amulti-user VR game, etc.), and/or any other objects portrayed in virtualapplications.

The virtual platform preferably includes a tracking subsystem configuredto determine and/or track a user's location (e.g., in a room or otherpredefined space, relative to a virtual object, etc.), furtherpreferably a location of a particular region of the user's body (e.g.,the second body region(s), the user's hands, the user's fingers, theuser's torso, the user's head, the user's arm(s), the user's leg(s), theuser's feet, etc.). The tracking system preferably implements one ormore of the following types of tracking, but can additionally oralternatively implement any other suitable tracking: wireless tracking,optical tracking (e.g., with markers, without markers, etc.), inertialtracking, acoustic tracking, magnetic tracking, tracking with sensorfusion (e.g., merging inertial and optical tracking), and/or any othertracking or combination of tracking types. In preferred variations, forinstance, the tracking subsystem is configured to track one or more ofthe user's fingers, but can additionally or alternatively be configuredto track any suitable body regions.

The virtual platform can include any number of sensors configured totrack the user, such as, but not limited to, any or all of: infraredsensors (e.g., infrared cameras), cameras of visible range, stereocameras, depth cameras, accelerometers, gyroscopes, inertial measurementunits (IMUs), and/or any other suitable sensors for tracking.Additionally or alternatively, any or all of the haptic device and/orsensors of the haptic device can be implemented in tracking. In somevariations, for instance, the haptic device includes markers (e.g., IRmarkers) involved in tracking the haptic device, which can optionally beused to determine the location of a body region (e.g., proximal to thehaptic device, at a predetermined and/or calculatable offset from thehaptic device, etc.) of the user. Additionally or alternatively, anyother sensors of the haptic device (e.g., onboard accelerometer, onboardmotion sensor, onboard gyroscope, onboard IMU, etc.) can be used intracking the haptic device. Further additionally or alternatively, anysuitable components of the virtual platform (e.g., controllers, displaydevices, etc.) can be part of, involved with, and/or otherwise interfacewith the tracking subsystem. In some variations, for instance, thecontrollers include one or markers or sensors involved in tracking.

In some variations, tracking the user and/or one or more body regions ofthe user is performed with visible light and/or infrared light detection(e.g., with a wristband having infrared reflecting areas). In specificexamples, a wristband is tracked. In additional or alternative specificexamples, the user's hands and/or fingers are tracked.

Additionally or alternatively, tracking the user and/or body regions ofthe user can involve inertial measurement unit (IMU) sensor integrationto determine an orientation of the device. In specific examples, this iscombined with another tracking process to determine a location/positionof the device.

Further additionally or alternatively, tracking the user and/or bodyregions of the user can involve using a VR and/or AR system's remotecontroller tracking and/or other device tracking (e.g., a hand-trackingglove used in combination with a wristband device as described above).In specific examples, for instance, if the system knows where and inwhat orientation the handheld controller and/or other device is, thelocation of the device (e.g., wristband) and/or body regions (e.g.,fingers, hand, etc.) can be determined.

Further additionally or alternatively, acoustic tracking can beimplemented, wherein the device (e.g., wristband) emits sounds (e.g.,audible sounds, inaudible sounds, etc.) which are detected by a set ofsensors (e.g., multiple audio sensors such as microphones), whichenables a determination of the position and orientation of the device.

Further additionally or alternatively, a combination of any of the above(e.g., IMU sensor integration combined with another tracking process),and/or any other suitable tracking process can be used.

The virtual platform can additionally or alternatively include any othersensors (e.g., as described above for the haptic device) configured todetect other information, such as, but not limited to, any or all of:audio from the user, temperature, humidity, location (e.g., GPScoordinates), and/or any other suitable information.

The virtual platform can include any number of input devices configuredto receive inputs from the user, such as one or more controllers (e.g.,handheld controllers, wireless controllers, wired controllers,joysticks, etc.). In some variations, for instance, the virtual platformincludes a pair of handheld controllers (e.g., as shown in FIG. 7 )which the user uses during use of the virtual platform (e.g., tonavigate and/or interact with the virtual environment). Additionally oralternatively, the virtual platform can include other suitablecontrollers, the controllers can be integrated with the haptic device,the virtual platform can be absent of controllers, and/or the virtualplatform can be otherwise configured.

The virtual platform can include and/or interface with a display and/ordisplay device, which functions to provide visual content including thevirtual objects and/or any other form of content (e.g., audio) to theuser. The display can be part of a headset, glasses (e.g., AR glasses),and/or be part of or include any other suitable display (e.g., screen)or display device.

In a first set of variations, the virtual platform includes a VRplatform configured for gaming and/or any other suitable applications,wherein the VR platform includes a set of handheld controllers and aheadset configured to enable a user to interact with a virtualenvironment including a set of virtual objects. In specific examples,the user wears a set of wristband haptic devices which provide hapticstimulation based on the user's interaction with the virtual objects(e.g., at the user's fingertips and translated into haptic stimulationat the wrist based on sensory displacement).

In a second set of variations, the virtual platform includes an ARplatform configured to create and manage virtual objects which areintegrated with real-world imagery.

Additionally or alternatively, the virtual platform can be otherwiseconfigured and/or include any other components and/or combination ofcomponents.

3.3 System: Optional Components

The system can include and/or interface with a control module (e.g.,onboard the haptic device, onboard the virtual platform, onboard a userdevice, remote from the haptic device and/or virtual platform,distributed among multiple devices and/or components and/or computingsystems, etc.), which functions to apply stimulation through the hapticactuators at the haptic device. The control module can include aprocessing subsystem and/or computing subsystem (e.g., onboard thehaptic device, distributed among multiple components and/or computingsystems such as a remote computing system, etc.) wherein the processingsubsystem and/or computing subsystem determines one or more stimulationpatterns, stores one or more stimulation patterns, monitors systemperformance, implements a fail-safe (e.g., power shut-off in the eventof overheating or stimulation pattern parameter above a predeterminedthreshold, alarm, etc.), and/or performs any other suitable function.Determining a stimulation pattern can include any or all of: determininga new stimulation pattern (e.g., based on an algorithm, based on amachine learning model, etc.), selecting a stimulation pattern (e.g.,from a lookup table, from a library, from a record of previously appliedstimulation patterns, etc.), determining a set of parameters associatedwith a stimulation pattern (e.g., a set of weights for a stimulationpattern algorithm, an amplitude a stimulation, a frequency ofstimulation, etc.), and/or any other suitable stimulation pattern and/orparameter(s) associated with a stimulation pattern. The control moduleis preferably in communication with a control module and/or computingsubsystem (e.g., remote computing system, local computing system, etc.)of the virtual platform, wherein the control module receives informationassociated with the virtual object(s) (e.g., parameters as describedbelow), wherein a processing subsystem and/or computing subsystem of thecontrol module prescribes the stimulation pattern(s) in accordance withthe virtual object information. Additionally or alternatively, thecontrol module can determine the virtual object information and/or beotherwise configured and/or distributed.

In a first variation, the system 100 includes a control module at leastpartially arranged onboard the haptic device, wherein the control moduledetermines and prescribes a stimulation pattern to be applied at any orall of the haptic actuators onboard the haptic device. The onboardcontrol module is preferably in communication with a control moduleand/or computing subsystem of the virtual platform, such that thecontrol module receives one or more inputs including information aboutthe virtual objects to be represented through haptic stimulation.Additionally or alternatively, the control module can be partially orfully arranged remotely, partially or fully arranged at the virtualplatform, distributed among multiple components, and/or otherwisearranged.

The system 100 can optionally include and/or be configured to interfacewith a user device and/or a client application executing on a userdevice. The client application can optionally enable a user to selectone or more operational parameters of the system, perform informationprocessing, serve as part of the virtual platform, receive sensorinformation, and/or otherwise be implemented. Examples of a user deviceinclude a tablet, smartphone, mobile phone, laptop, watch, wearabledevice (e.g., glasses), or any other suitable user device. The userdevice can include power storage (e.g., a battery), processing systems(e.g., CPU, GPU, memory, etc.), user outputs (e.g., display, speaker,vibration mechanism, etc.), user inputs (e.g., a keyboard, touchscreen,microphone, etc.), a location system (e.g., a GPS system), sensors(e.g., optical sensors, such as light sensors and cameras, orientationsensors, such as accelerometers, gyroscopes, and altimeters, audiosensors, such as microphones, etc.), data communication system (e.g., aWiFi module, BLE, cellular module, etc.), and/or any other suitablecomponent(s).

Additionally or alternatively, the system 100 can include any or all of:a power source, a communication module (e.g., a wireless communicationmodule, Wifi chip, Bluetooth chip, to establish communication betweenthe haptic device and the virtual platform, etc.), and/or any othersuitable components.

In a first variation of the system 100, the system includes a hapticdevice configured to be worn on a wrist of the user, wherein thewristband device is configured to interface with a virtual platform, andincludes a set of haptic actuators arranged around at least a partialcircumference of the user's wrist, and optionally any other componentsonboard the wristband and/or in communication with the wristband, suchas but not limited to, a control module, power source, sensors, markersfor the tracking subsystem, and/or any other suitable components. In apreferred example, the wristband device includes 5 actuators (e.g.,LRAs) with 1 representing each of the user's fingers. Additionally oralternatively, the wristband device can have less than 5 actuators(e.g., 4 actuators as shown in FIG. 10 ), more than 5 actuators (e.g.,10 actuators for all of the user's fingers, etc.), and/or any othersuitable number of actuators. The user can optionally wear multiplewristband devices (e.g., 1 for each wrist, multiple going up the user'sarm, etc.).

In a second variation, the haptic device includes a vest including anarray of haptic actuators (e.g., in a uniform planar array, in a seriesof strips, etc.).

Additionally or alternatively, the system 100 can include any othersuitable components.

4. Method 200

As shown in FIG. 2 , a method 200 for haptic stimulation includes:receiving a set of inputs S210; determining one or more parametersassociated with a virtual object based on the set of inputs S220; andprescribing a stimulation pattern to one or more haptic actuators of ahaptic device S250. Additionally or alternatively, the method 200 caninclude any or all of: presenting information to a user S230; selectingone or more haptic actuators of a haptic device based on theparameter(s) S240; repeating any or all of S210-S2150; and/or any othersuitable processes performed in any suitable order. Further additionallyor alternatively, the method 200 can include any or all of the methods,processes, embodiments, and/or examples described in any or all of: U.S.application Ser. No. 14/750,626, filed 25 Jun. 2015, U.S. applicationSer. No. 15/661,934, filed 27 Jul. 2017, U.S. application Ser. No.15/795,054, filed 26 Oct. 2017, and U.S. application Ser. No.17/033,433, filed 25 Sep. 2020, each of which is incorporated herein inits entirety by this reference.

The method 200 functions to convey information to a user through hapticstimulation, such as information associated with features of one or morevirtual objects in a virtual platform. Additionally or alternatively,the method 200 can function to: provide haptic stimulation in accordancewith sensory displacement, provide haptic stimulation in accordance withsensory addition, and/or provide haptic stimulation based on any othersuitable information.

In a first set of variations, for instance, the method 200 functions toprovide haptic stimulation which represents any or all of the followinginformation associated with virtual objects in a virtual platform: alocation and/or proximity of the virtual object relative to the user(e.g., in the user's visual field, behind the user and/or outside of theuser's visual field, etc.); a degree of contact/force between the user(e.g., users' fingertips) and the virtual object (e.g., pressure/forceexerted on a target virtual object, a pressure/force exerted by targetvirtual objects “on” the user, etc.); an orientation of a virtualobject; a temperature of one or more virtual objects (e.g., indicatingdanger in a gaming application); one or more discrete states of a targetvirtual object (e.g., whether object is in contact with locations and/ordesired locations, whether a threshold is reached, etc.); a textureand/or other surface information of a virtual object; and/or any othersuitable information.

The method 200 is preferably performed at one or more computing systems,such as any or all of those described above (e.g., control module of thehaptic device, computing system of the virtual platform, remotecomputing system, etc.). Additionally or alternatively, the method 200can be performed at any location(s) with any suitable component(s)and/or system(s).

4.1 Method: Receiving a Set of Inputs S210

The method 200 includes receiving a set of inputs S210, which functionsto receive information with which to determine the haptic stimulationspatterns to be applied to the user. Additionally or alternatively, S210can function to receive information with which to select which actuatorsand/or haptic device(s) to provide haptic stimulation from, and/or canperform any other suitable information.

S210 is preferably performed initially in the method 200, and furtherpreferably performed continuously (e.g., at a predetermined frequency,at random intervals, etc.) throughout the method 200 as the userinteracts with the virtual platform. Additionally or alternatively, S210can be performed in response to a trigger and/or at any suitable time(s)during the method 200.

The set of inputs in S210 are preferably received from at least thevirtual platform (e.g., a computing system involved in dynamicallyupdating the virtual environment of the virtual platform, a controller,a headset, etc.), and optionally additionally or alternatively from thehaptic device (e.g., a sensor system of the haptic device, a controlmodule of the haptic device, a communication subsystem of the hapticdevice, etc.), a computing system (e.g., remote computing system,computing system of the haptic device, etc.), a user device, and/or fromany other suitable components.

The set of inputs preferably includes information from the virtualplatform, further preferably information associated with one or morevirtual objects, which can be static, dynamic (e.g., moving, changingshape and/or size, changing state, etc.), and/or any combination. Theone or more virtual objects can be prescribed (equivalently referred toherein as target) virtual objects (e.g., objects of importance in agaming application, objects corresponding to characters in a game,surgical tool in a surgical simulation, anatomical region of interest ina surgical simulation, delicate anatomical region in a surgicalsimulation, etc.), dynamically determined virtual objects (e.g., basedon user input, dynamically updated as the virtual environment changes,based on proximity to the user, etc.), all virtual objects,randomly-determined virtual objects, and/or any other suitable virtualobjects.

The set of inputs preferably includes virtual object information fromthe virtual platform, further preferably spatial information associatedwith the virtual object.

The spatial information preferably includes a location (equivalentlyreferred to herein as a position) of the virtual object (e.g., a vectorposition, a location in geographical coordinates, a location and/orposition relative to a prescribed space, etc.), which can be used (e.g.,in S220) to determine a distance parameter of the virtual objectrelative to the user. Additionally or alternatively, a distanceparameter can be directly received and/or otherwise determined.

The spatial information can additionally or alternatively includeorientation information (e.g., angle in degrees, angle in radians,multiple angles, etc.) associated with the virtual object, which can beused (e.g., in S220) to determine one or more angles of the virtualobject relative to the user. Additionally or alternatively, an angleparameter can be directly received and/or otherwise determined.

The information associated with the virtual objects can additionally oralternatively include information associated with one or more featuresand/or parameters of virtual objects. The features can be predetermined(e.g., programmed, assigned, static, etc.), dynamically determined(e.g., prescribed and evolving as the virtual object moves and/orchanges), determined based on other information (e.g., location of thevirtual object, location of the user, proximity between the virtualobject and the user, etc.), and/or any combination.

The features and/or parameters can optionally include, for instance, aforce and/or pressure value associated with the virtual object and/orthe user, such as a force and/or pressure (e.g., in Newton, in psi, in anormalized value, in a percentage, in a unitless value, etc.) exerted ona virtual object (e.g., by the user when he is in contact with thevirtual object, by other virtual objects, based on informationassociated with the virtual object such as location and/or proximity toa virtual object or effect, etc.); a force and/or pressure exerted bythe virtual object (e.g., on a particular location, on a location of theuser(s), on a particular body region of the user, on a location of thehaptic device, etc.); a force and/or pressure of a virtual effect (e.g.,virtual rain, wind, etc.); and/or any other suitable information.

The features and/or parameters can additionally or alternatively includea texture associated with a virtual object (e.g., smooth surface, roughsurface, regular texture, irregular texture, fibers, hair, fur, fluffysurface, etc.), which can be represented through a tactile stimulationpattern in subsequent processes of the method (e.g., when user is incontact with the virtual object).

The features and/or parameters can additionally or alternatively includea temperature associated with a virtual object, such as a temperature ofa virtual object (e.g., in Fahrenheit, in Celsius, in a normalizedvalue, in a unitless value, as a percentage of maximum temperature, in aqualitative representation of “cold” vs. “normal” vs. “hot”, etc.). Thetemperature can optionally represent and/or be used to determine any orall of: a temperature of the virtual object relative to the virtualenvironment, a temperature of the virtual object relative to anothervirtual object, a temperature of the virtual object relative to adesired and/or predetermined temperature, and/or any other suitabletemperature.

The features and/or parameters can additionally or alternatively includea discrete state associated with the virtual object, such as, but notlimited to any or all of: whether a virtual object is a target object(e.g., object of interest, object most relevant to and/or closest to theuser, etc.) or not, whether a virtual object is within or outside thefield of view of the user, whether a virtual object's temperature is hotor cold (e.g., based on temperature information and a set ofthresholds), and/or any suitable states.

The features and/or parameters can additionally or alternatively includeone or more features not typically detectable and/or perceivable by ahuman, such as, but not limited to, echolocation (e.g., from a virtualbat), ultrasound, frequencies of sound outside of a detectable range,wavelengths of light outside of a visible range, electromagnetic waves,and/or any other suitable features representing any suitable phenomena.

The set of inputs received in S210 includes information associated withthe user, further preferably information received from a trackingsubsystem of a virtual platform. This can include, for instance, spatialinformation associated with the user, such as a location and/ororientation information of the user (e.g., as described above).

The tracking subsystem can collect information associated with thelocation and/or orientation of the haptic device (e.g., wristband hapticdevice with a set of markers, any haptic device with a set of markers,haptic device without markers, etc.); information associated with thelocation and/or orientation of the user himself (e.g., based on a visualtracking subsystem, for a user wearing markers, etc.); informationassociated with the location and/or orientation of a particular bodyregion of the user (e.g., fingers and/or fingertips, hands, etc.);information associated with one or more components of the virtualplatform such as location and/or orientation information of a headsetand/or set of controllers; and/or any other suitable information.

In preferred variations, for instance, a tracking subsystem of thevirtual platform collects location and orientation informationassociated with a haptic device (e.g., wristband device) coupled to theuser. The haptic device can be directly tracked (e.g., with a set ofmarkers on the haptic device, etc.); additionally or alternatively, alocation and/or orientation of the wristband can be tracked by directlytracking a set of controllers held by the user in his hands (e.g., andbased on a predetermined mapping from the controller position when heldto the wristband). In specific examples involving a wristband hapticdevice, the tracking subsystem collects location and orientationinformation of the wristband.

The information associated with the user can additionally oralternatively include a set of inputs collected from the user, such asbut not limited to: inputs received at one or more controllers of thevirtual platform (e.g., button presses, movement of controllers,selections made at controllers, etc.); audio inputs received from user(e.g., voice commands); and/or any other suitable inputs.

In a first variation of S210, S210 includes collecting a vector locationof a virtual object and one or more vector locations of the user (e.g.,of the haptic device(s), of controller(s), etc.). S210 furtherpreferably includes collecting an orientation associated with the user(e.g., angles of orientation of haptic device) and optionally anorientation associated with the virtual object. Additionally oralternatively, virtual object feature information, user inputs, and/orany other suitable information can be received.

4.2 Method: Determining One or More Parameters Associated with theVirtual Object Based on the Set of Inputs S220

The method 200 includes determining one or more parameters associatedwith the virtual object based on the set of inputs S220, which functionsenable the determination and/or selection of a stimulation pattern to beapplied to the user. Additionally or alternatively, S220 can function todetermine which of a set of multiple features of a virtual object torepresent in the stimulation pattern (e.g., by prioritizing a set ofmultiple parameters); characterize a relationship (e.g., in contact, notin contact, close to being in contact, etc.) between the user and avirtual object; determine one or more stimulation parameters (e.g.,frequency value, amplitude, duration, pulsing parameters, etc.) in astimulation pattern; and/or perform any other suitable function(s).

S220 is preferably performed in response to S210, wherein the parametersare determined from and/or equal to the information collected in S210.As such, S220 can optionally be performed continuously throughout themethod 200 and/or at any suitable time(s) during the method 200.Additionally or alternatively, S220 can be performed prior to S210and/or at any suitable time(s).

S220 preferably includes determining a proximity parameter between theuser and a virtual object (e.g., target object), wherein the proximityparameter can include any or all of: a distance between the virtualobject and one or more locations associated with and/or on the user(e.g., haptic device location, controller location, headset location,hand location, finger location, other body part location, averagedlocation, etc.); a discrete state of whether or not the user is incontact with the virtual object; an indication of whether the user isgetting closer to or farther away from a virtual object; and/or anyother proximity information.

The proximity parameter is preferably determined based on locationinformation collected in S210, such as vector locations of the user andvirtual objects. Additionally or alternatively, the proximity parametercan be determined based on orientation information (e.g., as describedabove) associated with the user and/or virtual object(s). Determiningthe proximity parameter (e.g., distance) can optionally include anynumber of mathematical (e.g., projecting position coordinates into aplace, calculating a vector between position coordinates, etc.),algorithmic, and/or machine learning processes.

In preferred variations, for instance, S220 includes determining adistance between a region of interest (e.g., fingertips, hand, anyregion, etc.) of the user and a virtual object (e.g., target object),and optionally an orientation the user relative to the virtual object,which can subsequently be used to determine whether or not the region ofinterest is in contact with the virtual object. Additionally oralternatively, it can be determined if any part of the user is incontact with the virtual object, if a particular point representation ofthe user (e.g., center of mass, center of gravity, etc.) is in contactwith a virtual object, if the haptic device and/or controller and/orheadset is in contact with a virtual object, how many virtual objectsthe user is in contact with, and/or any other distances can bedetermined.

Determining the proximity parameter (e.g., distance) can include (e.g.,in variations involving sensory displacement) determining one or moremappings S222 between system components (e.g., haptic devices, virtualplatform components, etc.) and/or body regions of the user. This canfunction, for instance, to enable the location and/or other informationassociated with a body region of the user which is not directly tracked.In variations involving wristband haptic devices, for instance, S220 caninclude (e.g., when determining if a user is touching a virtual objectwith his or her fingers) determining a mapping from the location and/ororientation of the wristband device (e.g., which provides the locationand/or orientation information in S210), such as in embodiments in whichthe wristband device is being tracked, to one or more fingers of theuser's hand(s), such that parameters associated with the interaction ofthe user's fingers and a virtual object can be determined and used todetermine a stimulation pattern. The mapping can be determineddynamically (e.g., based on supplementary information collected at acontroller and/or haptic device and/or headset, based on supplementaryinformation collected at one or more sensors, etc.), predetermined(e.g., based on predetermined distances between the user's body regionand the haptic device placement, based on aggregated and/or averagedistances between a user's body region and the haptic device placement,based on an anatomical average distance between a human's fingertips andwrist, etc.), and/or otherwise determined. The mapping(s) can bespecific to a particular user, aggregated for multiple users, and/orotherwise determined.

Additionally or alternatively the mapping can be between body regions(e.g., between fingers, between hands, between wrist and finger, etc.),between any suitable components and/or user body regions, and/or betweenany other locations.

S220 can additionally or alternatively include determining a proximityparameter (e.g., distance) of a virtual object to another virtualobject, a proximity parameter of a user relative to a predeterminedlocation, a proximity parameter of a virtual object relative to apredetermined location, and/or any other suitable proximity parameters.

The parameters can additionally or alternatively include any or all ofthe information described in S210, such as a texture parameter (e.g.,length of fur, smooth vs. rough, etc.), a temperature value, aparticular discrete state, and/or any other parameters.

In one variation of S220, S220 includes determining a set of proximityparameters between the fingers of a user and a virtual object, whereinthe proximity parameter is determined based on location information andoptionally orientation information associated with a wristband hapticdevice worn by the user. The proximity parameter for each fingerincludes a distance from the fingertip to the virtual object (e.g.,nearest surface of the virtual object), and is determined based on amapping from each of the fingertips of the wristband device (e.g., thepoints on the wristband device being tracked such as in a virtualplatform using IR sensing). In specific examples, these distances areused to determine whether or not each of the fingertips is in contactwith the virtual object. S220 can additionally include determining ifthere are parameters associated with features of the virtual object tobe conveyed to the user (e.g., upon contact with the user), such astexture parameters (e.g., fur length).

In a second variation of S220, S220 includes determining a temperatureparameter (e.g., temperature value) of a virtual object relative to theuser, wherein the temperature parameter can optionally take into accounta distance between the virtual object and the user (e.g., a hottemperature increases as the object gets closer to the user) and/orbetween the virtual object and another virtual object (e.g., as thevirtual object gets closer to a virtual sun).

Additionally or alternatively, S220 can be performed in any othersuitable way.

4.3 Method: Selecting One or More Haptic Actuators of the Haptic DeviceBased on the Parameter(s) S240

The method 200 preferably includes selecting one or more hapticactuators of the haptic device based on the parameter(s), whichfunctions to prescribe the location(s) at which haptic stimulation isapplied to the user. Additionally or alternatively, S240 can function toselect a haptic device at which to apply stimulation (e.g., leftwristband haptic device vs. right wristband haptic device) and/or canfunction to select any other component(s). Additionally oralternatively, in variations involving multiple haptic devices (e.g.,distributed among multiple users, coupled to a single user, etc.) themethod 200 can optionally function to select a subset of haptic devicesat which to provide haptic stimulation. Further additionally oralternatively, the method 200 can include selecting a particular userand/or subset of users (e.g., such as in a virtual reality gameinvolving multiple users) at which to apply haptic stimulation, and/orcan include any other suitable processes and/or selection of hapticactuators.

S240 is preferably performed in response to S220 and based on theparameters and/or their values. Additionally or alternatively, S240 canbe performed continuously and/or multiple times throughout the method,at other time(s), and/or at any time(s) during the method 200.

The haptic actuators and/or devices are preferably selected based on theparameters determined in S220, but can additionally or alternatively bepredetermined (e.g., all actuate) and/or determined based on anysuitable information.

In a first set of variations (e.g., as shown in FIG. 5 and FIGS. 9A-9B),all actuators associated with a region determined to be in contact witha virtual object are actuated (e.g., with the same stimulationparameters, with different stimulation parameters, etc.).

In a second set of variations, all actuators having a parametersatisfying thresholds and/or criteria are actuated, such as allactuators: above a predetermined threshold (e.g., temperature above atemperature threshold, etc.), below a predetermined threshold (e.g.,proximity threshold), associated with a particular discrete state,and/or any other suitable actuators.

In a third set of variations, only a single actuator and/orpredetermined subset of actuators on a haptic device is actuated at agiven time. This actuator/subset of actuators can be associated with amaximum value for a parameter, a minimum value for a parameter, and/orbe otherwise chosen.

Additionally or alternatively, haptic actuators can be otherwiseselected.

4.4 Method: Prescribing a Stimulation Pattern to the One or More HapticActuators S250

The method 200 includes prescribing a stimulation pattern to the one ormore haptic actuators S250, which functions to represent informationcollected above in the form of haptic stimulation.

S250 is preferably performed in response to S240, but can additionallyor alternatively be performed in response to S220, S210, continuouslyand/or multiple times throughout the method 200, and/or at any suitabletime(s).

The stimulation pattern is preferably determined based on the value ofthe one or more parameters determined in S220, wherein the values of theparameter can be used to select one or more parameters of thestimulation (e.g., frequency, amplitude, duration, etc.). Thestimulation parameters can be determined based on any or all of:predetermined assignments in a lookup table, dynamically determinedassignments (e.g., based on algorithms, decision trees, machine learningmodels, etc.), a combination of parameters associated with the virtualobject (e.g., temperature and texture and proximity), and/or any othersuitable information.

One or more parameters of the stimulation pattern can optionallyadditionally or alternatively be determined, in part or in full, basedon the type of parameter being represented by the haptic stimulation.Types of parameters can include, for instance, but are not limited to:proximity, pressure, force, temperature, texture, location (e.g.,elevation), a discrete state (e.g., of the virtual object, of a featureof the virtual object such as color, etc.), any or all of the parametertypes described above, and/or any other suitable parameter types. Insome variations, for instance, the type of stimulation applied (e.g.,pulsing vs. continuous, single pulse vs. double pulse, etc.) isdetermined based on the type of features (e.g., temperature vs.proximity vs. pressure, etc.) being represented.

S250 can optionally include comparing one or more parameters with athreshold and/or set of criteria in order to determine a stimulationpattern. This can include determining, for instance, any or all of:whether or not an object is currently in contact with a predeterminedlocation (e.g., location of the user, location of another virtualobject, etc.); whether a threshold is close to being met and/or has beenmet and/or exceeded (e.g., whether a pressure exerted by the user on avirtual object may cause the object to “break”, whether a surgical tooland/or its virtual representation is within a predetermined distancethreshold of a delicate region of a virtual representation of apatient's body, etc.); whether or not a discrete state applies to thevirtual object or not; and/or any other thresholds and/or criteria canbe implemented.

One or more parameters of the stimulation pattern can optionally bedetermined, for instance, based on the distance between the user and thevirtual object, wherein the amplitude of stimulation provided at thehaptic actuator increases as the virtual object gets closer.Additionally or alternatively, all haptic actuators associated withcontact with the virtual object can vibrate with the same stimulationpattern.

In a first variation, for instance, the user wears a haptic wristbanddevice on each of her wrists, wherein each wristband device includes anarray of one or more haptic actuators (equivalently referred to hereinas vibrational units) at positions around the circumference of thewrist. In one example embodiment (e.g., as shown in FIGS. 3A-3B, 4A-4B,and 5 ), a user wears two such wristbands each containing fivevibrational linear resonant actuators spread around the user's wrists.In an example virtual environment, the user interacts with a virtualballoon, which has the ability to pop if a user exerts too much forceupon it with the user's hands. In this embodiment, the target object isthe virtual balloon, the actuators on the left wristband correspond thefive fingertips of the user's left hand, and the actuators on the righthand to the five fingers of the user's right hand. In this example, eachactuator vibrates in an inverse relationship between the distance ofeach corresponding fingertip of the user to the distance of thecorresponding closest point on the balloon. As parts on the balloon getcloser to each fingertip, the corresponding actuator vibrates morestrongly. Upon making successful virtual contact between a fingertip anda point on the balloon, one or more actuators may vibrate with aparticular pattern (e.g., particular pulsing pattern, particularduration, particular frequency, particular amplitude, etc.) to indicatethat contact has been successfully made. Once the target object is indirect contact with the desired body locations—the user's fingers—thevibrational strength of each actuator may then be mapped to the pressurebeing exerted by the user on the balloon. The more pressure beingexerted by a particular finger may be mapped to a stronger vibration.This information can also be used to interact with the object in variousways, such as, but not limited to any or all of: deforming it, scalingit, punching it, zooming in and out, displacing it, and/or otherwisealtering the virtual object (e.g., as in S230). If a virtual thresholdis exceeded (or about to be exceeded), one or more actuators can vibratewith a second particular pattern indicating a potential state change(e.g. balloon is about to or has popped) to the user.

In an alternative embodiment still involving wristband haptic devices,the actuators are mapped to five locations around the user'scorresponding wrists (rather than mapped to the user's fingers). In thisembodiment, the actuators in either wristband vibrate to the spatialorientation of the balloon relative to the user's wrists in an inverserelationship with distance. For example, if the balloon is located tothe right of the user's right hand, the actuators representing theuser's right side of the right wrist will vibrate the most strongly,while all other actuators will vibrate with a lesser amplitude and/orother parameter, or not at all. If the balloon is located between theuser's hands, the actuators representing the inner wrists (closest tothe balloon) will vibrate most strongly, while the actuatorsrepresenting the outer wrists will vibrate less so (or not at all).

Additionally or alternatively, a haptic pattern (e.g., as describedabove and/or below, other than what is described above and/or below,etc.) can be applied to a user if an object is not grabbable (e.g., andthe user tries to grab it). This haptic pattern is preferably differentthan those implemented for a grabbable object such that the user canidentify that the object is not grabbable, but can additionally oralternatively include the same haptic patterns as those used otherwise.

In a second variation, the actuators may represent temperature ratherthan pressure, such that the stronger the vibration means a hotter orcolder temperature for a virtual object that is in virtual contact witha user's fingertips.

In a third variation, the stimulation pattern can convey textures ratherthan strictly vibration amplitude or strength, which can be achieved viamodulating electrical waveforms applied to the device. In this example,different sensations could be induced depending on the object that is invirtual contact with the user's fingertips. Fingertips being movedacross a virtual balloon may provide a different stimulus than fingersbeing moved across a virtual stuffed animal.

In a fourth variation (e.g., as shown in FIGS. 4A-4B), additional oralternative to those described above, haptic actuators in a hapticwristband vibrate around the wrist(s) of the user in the direction avirtual object is located, which is preferably implemented through oneor more illusion algorithms—but can additionally or alternatively beimplemented through any other suitable algorithm(s)—with the overallstrength of the vibration representing (e.g., correlating to) distance.

In a fifth variation, additional or alternative to those describedabove, one or more haptic devices and/or haptic device actuatorsvibrates at a distinct or otherwise recognizable vibration when thevirtual object is “grasped” by (virtually makes contact with) the user'shand.

In a sixth variation, additional or alternative to those describedabove, a wristband haptic device with 5 actuators, each representing afinger, is used as the haptic device. In specific examples, the strengthof each actuator represents how far the finger is to the actuator, and aparticular haptic pattern (e.g., for each finger, for all fingers, for asubset of fingers, etc.) can be implemented when the object is aboutwithin grasp, and a final pattern can be implemented when the object isactually grasped. From there, one could optionally switch to a modewhere the vibration strength represents the force being exerted on theobject by each finger.

Additionally or alternatively, S250 can be otherwise performed.

4.5 Method: Presenting Information to the User S230

The method 200 can optionally include presenting information to theuser, which can function to provide feedback to the user. Theinformation presented can include any or all of: visual feedback (e.g.,presented at the virtual platform), audio feedback (e.g., beeping),tactile feedback (e.g., to indicate that the user needs to adjust one ormore components, etc.).

S230 is preferably performed in response to S250 and/or S, wherein theinformation presented to the user includes visual feedback correspondingto the stimulation pattern, such as an indication of the amplitude ofstimulation being applied. S230 can additionally or alternatively beperformed continuously throughout the method 200 and/or at any othersuitable time(s).

Additionally or alternatively, S230 can include portraying how thevirtual object is manipulated (e.g., moved, squeezed, expanded,indented, pulled, etc.) based on the user's interaction with it.

In a first set of variations, S230 includes presenting a visualindicator at the virtual platform indicating any or all of: whether ornot haptic stimulation is being applied, a feature of the hapticstimulation being applied (e.g., how much, how large of an amplitude,how high of a frequency, etc.), a location of where the hapticstimulation is being applied and/or what body region it corresponds to(e.g., which hand, which actuator, which finger, etc.), a change in thestimulation a value of a parameter determined in S220, and/or any othersuitable information.

In a set of specific examples (e.g., as shown in FIGS. 6A-6B), an object(e.g., sphere) shows a stimulation amount, wherein a bigger spherecorresponds to a larger amplitude of stimulation.

4.6 Method: Repeating Any or All of S210-S250

The method 200 can optionally include repeating any or all of the aboveprocesses, in any suitable order, such as throughout the duration of theuser's use of the virtual platform. Additionally or alternatively, theprocesses of the method 200 can be otherwise performed.

5. Variations

In a first variation of the method 200, the method includes receiving aset of inputs S210, wherein the set of inputs includes inputs from avirtual platform associated with one or more virtual objects of thevirtual platform (e.g., vector location of virtual object, features ofvirtual object, orientation of a virtual object, etc.) and inputsassociated with a location of the user and/or an orientation of theuser, and optionally one or more user inputs (e.g., at a controller ofthe virtual platform); determining one or more parameters associatedwith the virtual object based on the set of inputs S220, such as any orall of: one or more proximity parameters between the user and thevirtual object (e.g., between the fingers of the user and the virtualobject, between the haptic device and/or haptic actuators and thevirtual object, etc.); a feature of the virtual object to be conveyed tothe user (e.g., upon contact with the user), such as texture parameters(e.g., fur length) and/or temperature parameters; a discrete state ofthe virtual object; and/or any other suitable parameters; selecting oneor more haptic actuators based on the parameter(s) S240; prescribing astimulation pattern to the selected haptic actuators S250; optionallypresenting information to the user S230 (e.g., based on thestimulation); optionally repeating any or all of the processes; and/orany other suitable processes performed in any suitable order.

In one embodiment of this variation as shown in FIGS. 9A-9B, a proximityparameter (e.g., distance) is determined between the user's fingertipsand a virtual object, wherein if contact is made, stimulation is appliedat the corresponding actuator. In a first example of this embodiment,Fingers 2, 3, and 4 are each determined to be in contact with thevirtual object (e.g., have a distance parameter less than or equal to0), which causes Actuators 2, 3, and 4 to all be actuated with the samestimulation pattern (e.g., same stimulation parameters). Actuators 1 and5 (corresponding to pinky and thumb, respectively) on opposing side ofwristband not actuated. In a second example of this embodiment, Fingers2, 3, and 4 are each determined to be in contact with the virtual object(e.g., have a distance parameter less than or equal to 0) with Finger 3closest (e.g., most overlapping), followed by Finger 4, followed byFinger 2. This results in Actuators 2, 3, and 4 all being actuated withdifferent stimulation patterns (e.g., decreasing amplitude from Finger 3to Finger 4 to Finger 2). Actuators 1 and 5 (corresponding to pinky andthumb, respectively) on opposing side of wristband not actuated. In athird example of this embodiment, Fingers 2, 3, and 4 each determined tobe in contact with the virtual object (e.g., have a distance parameterless than or equal to 0) with Finger 3 closest (e.g., most overlapping),followed by Finger 4, followed by Finger 2, which causes Actuators 2, 3,and 4 to all be actuated with different stimulation patterns (e.g.,decreasing amplitude from Finger 3 to Finger 4 to Finger 2). Actuators 1and 5 (corresponding to pinky and thumb, respectively) on opposing sideof wristband are also actuated (e.g., both with less amplitude thanActuator 2 and Actuator 5 with greater amplitude than Actuator 1; bothwith a stimulation pattern to indicate distance below a threshold butnot touching, such as a different frequency; etc.). Additionally oralternatively, the actuators can be otherwise actuated.

In a second embodiment of this variation involving a proximityparameter, the distance between the wristband (and/or its hapticactuators) and the virtual object is used to determine haptic actuatorsto be activated and/or the corresponding stimulation pattern(s).

In a third embodiment of this variation, one or more features of thevirtual object (e.g., temperature, texture, etc.) are used to determinethe stimulation pattern and can be determined based on the proximityparameter (e.g., texture stimulated only when in contact, temperatureincreasing with decreasing distance, etc.), determined and/or stimulatedindependently of proximity, and/or otherwise used.

Additionally or alternatively, the method 200 can include any othersuitable processes.

Although omitted for conciseness, the preferred embodiments includeevery combination and permutation of the various system components andthe various method processes, wherein the method processes can beperformed in any suitable order, sequentially or concurrently.

Embodiments of the system and/or method can include every combinationand permutation of the various system components and the various methodprocesses, wherein one or more instances of the method and/or processesdescribed herein can be performed asynchronously (e.g., sequentially),contemporaneously (e.g., concurrently, in parallel, etc.), or in anyother suitable order by and/or using one or more instances of thesystems, elements, and/or entities described herein. Components and/orprocesses of the following system and/or method can be used with, inaddition to, in lieu of, or otherwise integrated with all or a portionof the systems and/or methods disclosed in the applications mentionedabove, each of which are incorporated in their entirety by thisreference.

As a person skilled in the art will recognize from the previous detaileddescription and from the figures and claims, modifications and changescan be made to the preferred embodiments of the invention withoutdeparting from the scope of this invention defined in the followingclaims.

1. A system for providing haptic information associated with a set ofvirtual objects in at least one of a virtual and augmented realityplatform, the system comprising: a haptic stimulation device configuredto be coupled to the user at a first body region, wherein the hapticstimulation device comprises: a set of multiple haptic actuators, eachof the set of multiple haptic actuators configured to apply hapticstimulation to a skin surface of the user; a processing subsystem incommunication with the at least one of a virtual and augmented realityplatform, the processing system configured to determine a hapticstimulation pattern to be applied at the set of multiple hapticactuators, wherein the processing system: receives informationassociated with a first location, the first location corresponding to alocation of a virtual object of the set of virtual objects; receivesinformation associated with a second location, the second locationcorresponding to a location of a second body region of the user, thesecond body region separate and distinct from the first body region;determining a proximity metric based on the first location and thesecond location; selecting an actuator from the set of multipleactuators based on the proximity metric; and actuating the actuator.